Conducting sparse feature selection on arbitrarily long phrases in text corpora with a focus on interpretability

摘要

We propose a general framework for topic-specific summarization of large textcorpora, and illustrate how it can be used for analysis in two quite differentcontexts: an OSHA database of fatality and catastrophe reports (to facilitatesurveillance for patterns in circumstances leading to injury or death) andlegal decisions on workers' compensation claims (to explore relevant case law).Our summarization framework, built on sparse classification methods, is acompromise between simple word frequency based methods currently in wide use,and more heavyweight, model-intensive methods such as Latent DirichletAllocation (LDA). For a particular topic of interest (e.g., mental healthdisability, or chemical reactions), we regress a labeling of documents onto thehigh-dimensional counts of all the other words and phrases in the documents.The resulting small set of phrases found as predictive are then harvested asthe summary. Using a branch-and-bound approach, this method can be extended toallow for phrases of arbitrary length, which allows for potentially richsummarization. We discuss how focus on the purpose of the summaries can informchoices of regularization parameters and model constraints. We evaluate thistool by comparing computational time and summary statistics of the resultingword lists to three other methods in the literature. We also present a new Rpackage, textreg. Overall, we argue that sparse methods have much to offer textanalysis, and is a branch of research that should be considered further in thiscontext.